Maximal Information Divergence from Statistical Models Defined by Neural Networks
نویسندگان
چکیده
We review recent results about the maximal values of the Kullback-Leibler information divergence from statistical models defined by neural networks, including näıve Bayes models, restricted Boltzmann machines, deep belief networks, and various classes of exponential families. We illustrate approaches to compute the maximal divergence from a given model starting from simple subor super-models. We give a new result for deep and narrow belief networks with finite-valued units.
منابع مشابه
' & $ % Information Geometric Measurements of Generalisation Information Geometric Measurements of Generalisation
Neural networks can be regarded as statistical models, and can be analysed in a Bayesian framework. Generalisation is measured by the performance on independent test data drawn from the same distribution as the training data. Such performance can be quantiied by the posterior average of the information divergence between the true and the model distributions. Averaging over the Bayesian posterio...
متن کاملGyroscope Random Drift Modeling, using Neural Networks, Fuzzy Neural and Traditional Time- series Methods
In this paper statistical and time series models are used for determining the random drift of a dynamically Tuned Gyroscope (DTG). This drift is compensated with optimal predictive transfer function. Also nonlinear neural-network and fuzzy-neural models are investigated for prediction and compensation of the random drift. Finally the different models are compared together and their advantages a...
متن کاملPareto Optimization of Two-element Wing Models with Morphing Flap Using Computational Fluid Dynamics, Grouped Method of Data handling Artificial Neural Networks and Genetic Algorithms
A multi-objective optimization (MOO) of two-element wing models with morphing flap by using computational fluid dynamics (CFD) techniques, artificial neural networks (ANN), and non-dominated sorting genetic algorithms (NSGA II), is performed in this paper. At first, the domain is solved numerically in various two-element wing models with morphing flap using CFD techniques and lift (L) and drag ...
متن کاملComparing Prediction Power of Artificial Neural Networks Compound Models in Predicting Credit Default Swap Prices through Black–Scholes–Merton Model
Default risk is one of the most important types of risks, and credit default swap (CDS) is one of the most effective financial instruments to cover such risks. The lack of these instruments may reduce investment attraction, particularly for international investors, and impose potential losses on the economy of the countries lacking such financial instruments, among them, Iran. After the 2007 fi...
متن کاملExtracting Material Information from the CT Numbers by Artificial Neural Networks for Use in the Monte Carlo Simulations of Different Tissue Types in Brachytherapy
Background: The artificial neural networks (ANNs) are useful in solving nonlinear processes, without the need for mathematical models of the parameters. Since the relationship between the CT numbers and material compositions is not linear, ANN can be used for obtaining tissue density and composition.Objective: The aim of this study is to utilize ANN for determination of the composition and mass...
متن کامل